Multi-source data encoding, transmission and decoding using Slepian-Wolf codes based on channel code partitioning

ABSTRACT

System and method for designing Slepian-Wolf codes by channel code partitioning. A generator matrix may be partitioned to generate a plurality of sub-matrices corresponding respectively to a plurality of correlated data sources. The partitioning may be performed in accordance with a rate allocation among the plurality of correlated data sources. A corresponding plurality of parity matrices may then be generated based respectively on the sub-matrices, where each parity matrix is useable to encode correlated data for a respective correlated data source.

FIELD OF THE INVENTION

The present invention relates to the field of informationcoding/decoding, and more particularly to a system and method fordesigning Slepian-Wolf codes for distributed source encoding/decoding.

DESCRIPTION OF THE RELATED ART

Issues related to distributed lossless compression of correlated sourcesare relevant for a wide variety of applications, such as distributedsensor networks and multi-source video distribution, both wired andwireless, coding for relay channels, and digital communications, amongothers. Distributed source coding (DSC), whose theoretical foundationwas laid by Slepian and Wolf as early as 1973 (see D. Slepian and J. K.Wolf, “Noiseless coding of correlated information sources,” IEEE Trans.On Information Theory, vol. IT-19, pp. 471-480, July 1973, incorporatedby reference herein.), refers to the compression of the outputs of twoor more physically separated sources that do not communicate with eachother (hence distributed coding). These sources send their compressedoutputs to a central point (e.g., the base station) for joint decoding.DSC is related to the well-known “CEO problem” (in which a source isobserved by several agents, who send independent messages to anotheragent (the chief executive officer (CEO)), who attempts to recover thesource to meet a fidelity constraint, where it is usually assumed thatthe agents observe noisy versions of the source, with the observationnoise being independent from agent to agent), and is part of networkinformation theory.

Compressing two distinct signals by exploiting their correlation cancertainly provide a benefit in total rate cost. Moreover, Slepian andWolf showed that lossless compression of two separate sources can be asefficient as if they are compressed together as long as joint decodingis done at the receiver. Several successful attempts of constructingpractical coding schemes that exploit the potential of the Slepian-Wolf(SW) theorem have been developed. See, e.g., S. S. Pradhan and K.Ramchandran, “Distributed source coding using syndromes (DISCUS): designand construction,” Proc. DCC-1999, Data Compression Conference, pp.158-167, Snowbird, Utah, March 1999; A. Liveris, Z. Xiong, and C.Georghiades, “Compression of binary sources with side information at thedecoder using LDPC codes,” IEEE Communications Letters, vol. 6, pp.440-442, October 2002; A. Liveris, Z. Xiong, and C. Georghiades,“Distributed compression of binary sources using convolutional paralleland serial concatenated convolutional codes,” Proc. DCC-2003, DataCompression Conference, pp. 193-202, Snowbird, Utah, March 2003; A.Aaron and B. Girod, “Compression of side information using turbo codes,”Proc. DCC-2002, Data Compression Conference, pp. 252-261, Snowbird,Utah, April 2002; J. Garcia-Frias and Y. Zhao, “Compression ofcorrelated binary sources using turbo codes,” IEEE CommunicationsLetters, vol. 5, pp. 417419, October 2001; and J. Bajcy and P. Mitran,“Coding for the Slepian-Wolf problem with turbo codes,” Proc. IEEEGlobecom-2001, vol. 2 pp. 1400-1404, San Antonio, Tex., November 2001,all of which are incorporated by reference herein. All these schemes,with the exception of that of Garcia-Frias and Zhao, are based onasymmetric codes (see, e.g., S. S. Pradhan and K. Ramchandran,“Generalized coset codes for symmetric distributed source coding,”included herewith as Appendix G); that is, they losslessly compress onesource, while the other source is assumed to be perfectly known at thedecoder side and is used as side information.

Thus, for two discrete, memoryless, identically distributed sources Xand Y encoded separately at rates R₁ and R₂, respectively, these codesattempt to reach the two corner points on the Slepian-Wolf (SW) bound:(R₁,R₂)=(H(X),H(Y|X)) and (R₁,R₂)=(H(Y),H(X|Y)). However, often it isdesirable to vary the rates of individual encoders while keeping thetotal sum-rate constant. One technique for achieving this is timesharing. However, time sharing might not be practical because itrequires exact synchronization among encoders.

A second technique is the source-splitting approach of Rimoldi andUrbanke (see B. Rimoldi and R. Urbanke, “Asynchronous Slepian-Wolfcoding via source-splitting”, Proc. ISIT-1997 IEEE Int. Symp.Information Theory, pp. 271, Ulm, Germany, June, 1997, incorporated byreference herein), which potentially reaches all points on the SW boundby splitting two sources into three subsources of lower entropy.Garcia-Frias and Zhao, in the reference cited above, proposed a systemconsisting of two different turbo codes which form a large turbo codewith four component codes. In the symmetric scenario suggested (wherethe rates of both encoders are the same), half of the systematic bitsfrom one encoder and half from the other are sent. Further, instead ofsyndrome bits, parity bits are sent.

Pradhan and Ramchandran have outlined a method for constructing a singlecode based on the syndrome technique, which achieves arbitrary rateallocation among the two encoders (see S. S. Pradhan and K. Ramchandran,“Generalized coset codes for symmetric distributed source coding,”included herewith as Appendix G; S. S. Pradhan and K. Ramchandran,“Distributed source coding: symmetric rates and applications to sensornetworks,” Proc. DCC-2000, Data Compression Conference, pp. 363-372,Snowbird, Utah, March 2000; and S. S. Pradhan and K. Ramchandran,“Distributed source coding using syndromes (DISCUS): design andconstruction,” Proc. DCC-1999, Data Compression Conference, pp. 158-167,Snowbird, Utah, March 1999, incorporated by reference herein.). Themethod constructs independent subcodes of the main code and assigns themto different encoders. Each encoder sends only partial information aboutthe source; by combining two received bitstreams, a joint decoder shouldperfectly reconstruct the sources. Since joint decoding is performedonly on a single code, if this code approaches the capacity of a channelthat models the correlation among the sources, the system will approachthe SW limit. Thus, an advantage of this approach is the need of onlyone good channel code. Pradhan and Ramchandran also showed that thiscode does not suffer from any performance loss compared to thecorresponding asymmetric code. Moreover, any point on the SW bound canbe potentially reached without increasing the encoding/decodingcomplexity. Further, Pradhan and Ramchandran applied the method tocoding of two noisy observations of a source with scalar quantizer andtrellis codes.

While the theoretical limits and bounds of SW coding are wellunderstood, practical implementations and their actual performance andlimits of have not heretofore been determined.

SUMMARY OF THE INVENTION

One embodiment of the present invention comprises a system and methodfor implementing Slepian-Wolf codes by channel code partitioning.

In one embodiment, a generator matrix is partitioned to generate aplurality of sub-matrices corresponding respectively to a plurality ofcorrelated data sources. The partitioning may be performed in accordancewith a rate allocation among the plurality of correlated data sources. Acorresponding plurality of parity matrices may then be generated basedrespectively on the sub-matrices, where each parity matrix is useable toencode correlated data for a respective correlated data source.

BRIEF DESCRIPTION OF THE DRAWINGS

A better understanding of the present invention can be obtained when thefollowing detailed description of the preferred embodiment is consideredin conjunction with the following drawings, in which:

FIG. 1A illustrates a computer system suitable for implementing variousembodiments of the present invention;

FIG. 1B illustrates a network system comprising two or more computersystems that may implement an embodiment of the present invention;

FIG. 2 is an exemplary block diagram of the computer systems of FIGS. 1Aand 1B;

FIGS. 3A and 3B illustrate exemplary applications of the presentinvention, according to various embodiments;

FIG. 4 is a flowchart diagram illustrating one embodiment of a methodfor Slepian-Wolf coding;

FIGS. 5A-5C flowchart more detailed embodiments of the method of FIG. 4;

FIG. 6 illustrates Slepian-Wolf encoding, according to one embodiment;and

FIGS. 7 and 8 illustrate simulation results with IRA codes and turbocodes together with the SW bound, according to one embodiment.

While the invention is susceptible to various modifications andalternative forms, specific embodiments thereof are shown by way ofexample in the drawings and are herein described in detail. It should beunderstood, however, that the drawings and detailed description theretoare not intended to limit the invention to the particular formdisclosed, but on the contrary, the intention is to cover allmodifications, equivalents and alternatives falling within the spiritand scope of the present invention as defined by the appended claims.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS INCORPORATION BYREFERENCE

The following references are hereby incorporated by reference in theirentirety as though fully and completely set forth herein:

-   U.S. Provisional Application Ser. No. 60/______, titled    “Multi-Source Data Encoding, Transmission and Decoding”, filed Mar.    1, 2005, whose inventors are Vladimir M. Stankovic, Angelos D.    Liveris, Zixiang Xiong, Costas N. Georghiades, Zhixin Liu, Samuel S.    Cheng, and Qian Xu;-   U.S. patent application Ser. No. 11/______, titled “Data Encoding    and Decoding Using Slepian-Wolf Coded Nested Quantization to Achieve    Wyner-Ziv Coding”, filed Mar. 1, 2005, whose inventors are Zhixin    Liu, Samuel S. Cheng, Angelos D. Liveris, and Zixiang Xiong.-   D. Slepian and J. K. Wolf, “Noiseless coding of correlated    information sources,” IEEE Trans. On Information Theory, vol. IT-19,    pp. 471-480, July 1973.-   S. S. Pradhan and K. Ramchandran, “Distributed source coding using    syndromes (DISCUS): design and construction,” Proc. DCC-1999, Data    Compression Conference, pp. 158-167, Snowbird, Utah, March 1999.-   A. Liveris, Z. Xiong, and C. Georghiades, “Compression of binary    sources with side information at the decoder using LDPC codes,” IEEE    Communications Letters, vol. 6, pp. 440-442, October 2002.-   A. Liveris, Z. Xiong, and C. Georghiades, “Distributed compression    of binary sources using convolutional parallel and serial    concatenated convolutional codes,” Proc. DCC-2003, Data Compression    Conference, pp. 193-202, Snowbird, Utah, March 2003.-   A. Aaron and B. Girod, “Compression of side information using turbo    codes,” Proc. DCC-2002, Data Compression Conference, pp. 252-261,    Snowbird, Utah, April 2002.-   J. Garcia-Frias and Y. Zhao, “Compression of correlated binary    sources using turbo codes,” IEEE Communications Letters, vol. 5, pp.    417-419, October 2001.-   J. Bajcy and P. Mitran, “Coding for the Slepian-Wolf problem with    turbo codes,” Proc. IEEE Globecom-2001, vol. 2 pp. 1400-1404, San    Antonio, Tex., November 2001.-   B. Rimoldi and R. Urbanke, “Asynchronous Slepian-Wolf coding via    source-splitting”, Proc. ISIT-1997 IEEE Int. Symp. Information    Theory, pp. 271, Ulm, Germany, June, 1997.-   S. S. Pradhan and K. Ramchandran, “Distributed source coding:    symmetric rates and applications to sensor networks,” Proc.    DCC-2000, Data Compression Conference, pp. 363-372, Snowbird, Utah,    March 2000.-   H. Jin, A. Khandekar, and R McEliece, “Irregular repeat-accumulate    codes,” Proc. of 2nd International Symposium on Turbo codes and    related topics, pp. 1-8, September 2000.-   T. Berger, “Multiterminal source coding”, The Information Theory    Approach to Communications, G. Longo, Ed., New York:    Springer-Verlag, 1977.-   C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon limit    error-correcting coding and decoding: Turbo codes,” Proc. ICC'93,    IEEE Int. Conf. on Comm., pp. 1064-1070, Geneva, 1993.-   A. D. Wyner and J. Ziv, “The rate-distortion function for source    coding with side information at the decoder”, IEEE Trans. on    Information Theory, vol. IT-22, pp. 1-10, January 1976.-   J. Chou, S. S. Pradhan and K. Ramchandran, “Turbo and trellis-based    constructions for source coding with side information,” Proc.    DCC-2003, Data Compression Conference, pp. 33-42, Snowbird, Utah,    March 2003.-   T. Cover, “A proof of the data compression theorem of Slepian and    Wolf for ergodic sources”, IEEE Trans. on Information Theory, vol.    IT-21, pp. 226-228, March 1975.-   Y. Oohama, “The Rate-Distortion Function for the Quadratic Gaussian    CEO Problem,” IEEE Trans. on Information Theory, vol. 44, pp.    1057-1070, May 1998.-   T. S. Han and K. Kobayashi, “A unified achievable rate region for a    general class of multiterminal source coding systems,” IEEE Trans.    on Information Theory, vol. IT-26, pp. 277-288, May 1980.-   Y. Yang, S. Chen, Z. Xiong, and W. Zhao, “Wyner-Ziv coding based on    TCQ and LDPC codes,” Proc. of 37th Asilomar Conference on Signals,    Systems, and Computers, Pacific Grove, Calif., November 2003.

APPENDICES

This application includes eight appendices labeled A-H.

Appendix A comprises a paper titled: “Design of Slepian-Wolf Codes byChannel Code Partitioning” by Vladimir M. Stankovic, Angelos D. Liveris,Zixiang Xiong, and Costas N. Georghiades.

Appendix B comprises a paper titled: “On Code Design for theSlepian-Wolf Problem and Lossless Multiterminal Networks” by Vladimir M.Stankovic, Angelos D. Liveris, Zixiang Xiong, and Costas N. Georghiades.

Appendix C comprises a paper titled: “Slepian-Wolf Coded NestQuantization (SWC-NQ) for Wyner-Ziv Coding: Performance Analysis andCode Design” by Zhixin Liu, Samuel S. Cheng, Angelos D. Liveris &Zixiang Xiong.

Appendix D comprises a paper titled: “Slepian-Wolf Coded Nested LatticeQuantization for Wyner-Ziv Coding: High Rate Performance Analysis andCode Design” by Zhixin Liu, Samuel S. Cheng, Angelos D. Liveris &Zixiang Xiong.

Appendix E comprises a paper titled: “Layered Wyner-Ziv Video Coding” byQian Xu and Zixiang Xiong.

Appendix F comprises a paper titled: “A Turbo Code Tutorial” by WilliamE. Ryan.

Appendix G comprises a paper titled: “Generalized Coset Codes forSymmetric Distributed Source Coding” by S. Sandeep Pradhan and KannanRamchandran.

Appendix H comprises a paper titled: “Compression of Binary Sources withSide Information at the Decoder Using LDPC Codes” by Angelos D. Liveris,Zixiang Xiong and Costas N. Georghiades.

Terms

The following is a glossary of terms used in the present application:

Memory Medium—Any of various types of memory devices or storage devices.The term “memory medium” is intended to include an installation medium,e.g., a CD-ROM, floppy disks 104, or tape device; a computer systemmemory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM,Rambus RAM, etc.; or a non-volatile memory such as a magnetic media,e.g., a hard drive, or optical storage. The memory medium may compriseother types of memory as well, or combinations thereof. In addition, thememory medium may be located in a first computer in which the programsare executed, or may be located in a second different computer whichconnects to the first computer over a network, such as the Internet. Inthe latter instance, the second computer may provide programinstructions to the first computer for execution. The term “memorymedium” may include two or more memory mediums which may reside indifferent locations, e.g., in different computers that are connectedover a network.

Carrier Medium—a memory medium as described above, as well as signalssuch as electrical, electromagnetic, or digital signals, conveyed via acommunication medium such as a bus, network and/or a wireless link.

Programmable Hardware Element—includes various types of programmablehardware, reconfigurable hardware, programmable logic, orfield-programmable devices (FPDs), such as one or more FPGAs (FieldProgrammable Gate Arrays), or one or more PLDs (Programmable LogicDevices), such as one or more Simple PLDs (SPLDs) or one or more ComplexPLDs (CPLDs), or other types of programmable hardware. A programmablehardware element may also be referred to as “reconfigurable logic”.

Medium—includes one or more of a memory medium, carrier medium, and/orprogrammable hardware element; encompasses various types of mediums thatcan either store program instructions/data structures or can beconfigured with a hardware configuration program. For example, a mediumthat is “configured to perform a function or implement a softwareobject” may be 1) a memory medium or carrier medium that stores programinstructions, such that the program instructions are executable by aprocessor to perform the function or implement the software object; 2) amedium carrying signals that are involved with performing the functionor implementing the software object; and/or 3) a programmable hardwareelement configured with a hardware configuration program to perform thefunction or implement the software object.

Program—the term “program” is intended to have the full breadth of itsordinary meaning. The term “program” includes 1) a software programwhich may be stored in a memory and is executable by a processor or 2) ahardware configuration program useable for configuring a programmablehardware element.

Software Program—the term “software program” is intended to have thefull breadth of its ordinary meaning, and includes any type of programinstructions, code, script and/or data, or combinations thereof, thatmay be stored in a memory medium and executed by a processor. Exemplarysoftware programs include programs written in text-based programminglanguages, such as C, C++, Pascal, Fortran, Cobol, Java, assemblylanguage, etc.; graphical programs (programs written in graphicalprogramming languages); assembly language programs; programs that havebeen compiled to machine language; scripts; and other types ofexecutable software. A software program may comprise two or moresoftware programs that interoperate in some manner.

Hardware Configuration Program—a program, e.g., a netlist or bit file,that can be used to program or configure a programmable hardwareelement.

Graphical User Interface—this term is intended to have the full breadthof its ordinary meaning. The term “Graphical User Interface” is oftenabbreviated to “GUI”. A GUI may comprise only one or more input GUIelements, only one or more output GUI elements, or both input and outputGUI elements.

The following provides examples of various aspects of GUIs. Thefollowing examples and discussion are not intended to limit the ordinarymeaning of GUI, but rather provide examples of what the term “graphicaluser interface” encompasses:

A GUI may comprise a single window having one or more GUI Elements, ormay comprise a plurality of individual GUI Elements (or individualwindows each having one or more GUI Elements), wherein the individualGUI Elements or windows may optionally be tiled together.

A GUI may be associated with a graphical program. In this instance,various mechanisms may be used to connect GUI Elements in the GUI withnodes in the graphical program. For example, when Input Controls andOutput Indicators are created in the GUI, corresponding nodes (e.g.,terminals) may be automatically created in the graphical program orblock diagram. Alternatively, the user can place terminal nodes in theblock diagram which may cause the display of corresponding GUI Elementsfront panel objects in the GUI, either at edit time or later at runtime. As another example, the GUI may comprise GUI Elements embedded inthe block diagram portion of the graphical program.

Computer System—any of various types of computing or processing systems,including a personal computer system (PC), mainframe computer system,workstation, network appliance, Internet appliance, personal digitalassistant (PDA), television system, grid computing system, or otherdevice or combinations of devices. In general, the term “computersystem” can be broadly defined to encompass any device (or combinationof devices) having at least one processor that executes instructionsfrom a memory medium.

FIG. 1A—Computer System

FIG. 1A illustrates a computer system 82 operable to execute a programconfigured to implement various embodiments of the present invention. Asshown in FIG. 1A, the computer system 82 may include input devices suchas a mouse and keyboard, output devices (such as a display device andspeakers). The computer system 82 may also include a network interface(e.g., an Ethernet card) for communicating with other computers over anetwork.

The computer system 82 may include a memory medium(s) on which one ormore computer programs or software components according to any ofvarious embodiments of the present invention may be stored. For example,the memory medium may store one or more programs which are executable toperform any or all of the methods described herein. The memory mediummay also store operating system software, as well as other software foroperation of the computer system. Various embodiments further includereceiving or storing instructions and/or data implemented in accordancewith the foregoing description upon a carrier medium.

FIG. 1B—Computer Network

FIG. 1B illustrates a system including a first computer system 82 thatis coupled to a second computer system 90. The computer system 82 may beconnected through a network 84 (or a computer bus) to the secondcomputer system 90. The computer systems 82 and 90 may each be any ofvarious types, as desired. The network 84 can also be any of varioustypes, including a LAN (local area network), WAN (wide area network),the Internet, or an Intranet, among others. The computer systems 82 and90 may execute a program in a distributed fashion. For example, computer82 may execute a first portion of the program and computer system 90 mayexecute a second portion of the program.

As another example, computer 82 may display the graphical user interfaceof a program and computer system 90 may execute a portion of the programimplementing the main functionality (i.e., the non-user interfaceportion) of the program.

In one embodiment, the graphical user interface of the program may bedisplayed on a display device of the computer system 82, and theremaining portion of the program may execute on a device 190 connectedto the computer system 82. The device 190 may include a programmablehardware element and/or may include a processor and memory medium whichmay execute a real time operating system. In one embodiment, the programmay be downloaded and executed on the device 190. For example, anapplication development environment with which the program is associatedmay provide support for downloading a program for execution on thedevice in a real time system.

FIG. 2—Computer System Block Diagram

FIG. 2 is a block diagram representing one embodiment of the computersystem 82 and/or 90 illustrated in FIGS. 1A and 1B. It is noted that anytype of computer system configuration or architecture can be used asdesired, and FIG. 2 illustrates a representative PC embodiment. It isalso noted that the computer system may be a general purpose computersystem, a computer implemented on a card installed in a chassis, orother types of embodiments. Elements of a computer not necessary tounderstand the present description have been omitted for simplicity.

The computer may include at least one central processing unit or CPU(processor) 160 which is coupled to a processor or host bus 162. The CPU160 may be any of various types, including an x86 processor, e.g., aPentium class, a PowerPC processor, a CPU from the SPARC family of RISCprocessors, as well as others. A memory medium, typically comprising RAMand referred to as main memory, 166 is coupled to the host bus 162 bymeans of memory controller 164. The main memory 166 may store programsoperable to implement Slepian-Wolf coding according to variousembodiments of the present invention. The main memory may also storeoperating system software, as well as other software for operation ofthe computer system.

The host bus 162 may be coupled to an expansion or input/output bus 170by means of a bus controller 168 or bus bridge logic. The expansion bus170 may be the PCI (Peripheral Component Interconnect) expansion bus,although other bus types can be used. The expansion bus 170 includesslots for various devices such as described above. As shown, thecomputer comprises a network card 122 for communication with otherdevices, e.g., distributed sensor or video distribution systems, othercomputer systems, etc. The computer 82 further comprises a video displaysubsystem 180 and hard drive 182 coupled to the expansion bus 170.

As shown, a device 190 may also be connected to the computer. The device190 may include a processor and memory which may execute a real timeoperating system. The device 190 may also or instead comprise aprogrammable hardware element. The computer system may be operable todeploy programs according to various embodiments of the presentinvention to the device 190 for execution of the program on the device190.

FIGS. 3A and 3B—Exemplary Systems

Various embodiments of the present invention may be directed todistributed sensor systems, wireless or wired distributed video systems,or any other type of information processing or distribution systemsutilizing information coding, e.g., Slepian-Wolf coding.

For example, FIG. 3A illustrates one embodiment of a distributed sensorsystem. As FIG. 3A shows, a receiver 308 may be operable to receivesignals, e.g., correlated signals, from a plurality of sources,specifically from a plurality of sensors 306.

However, it is noted that the present invention can be used for aplethora of applications and is not limited to the above applications.In other words, applications discussed in the present description areexemplary only, and the present invention may be used in any of varioustypes of systems. Thus, the system and method of the present inventionis operable to be used in any of various types of applications,including the control of other types of devices such as multimediadevices, video devices, audio devices, telephony devices, Internetdevices, etc., as well as network control, network monitoring, financialapplications, entertainment, games, etc.

FIG. 4—Method for Slepian-Wolf Coding for Multiple Data Sources

FIG. 4 illustrates a method for realizing a system of L encoders and ajoint decoder for L correlated sources, where L is an integer greaterthan or equal to two, according to one set of embodiments. In variousembodiments, some of the method elements shown may be performedconcurrently, in a different order than shown, or may be omitted.Additional method elements may also be performed as desired. As shown,this method may operate as follows.

In 420, L codes (including L encoders and L corresponding decoders) arespecified given a generator matrix G. Embodiments of a method forspecifying the L codes, given the generator matrix G, are described morefully below.

In 430, data from the L correlated sources are encoded using the Lencoders, respectively. Embodiments of a method for performing theencoding are described more fully below.

In 440, the L encoded streams are decoded to recover informationgenerated by the L sources. Embodiments of a method for performing thedecoding are described more fully below.

FIG. 5A—Method for Specifying Slepian-Wolf Codes for Multiple DataSources

FIG. 5A illustrates one embodiment of a method for specifying L codesfor L correlated source streams. In 504, any point in the Slepian-Wolf(SW) admissible rate region may be selected. The point includes one ratevalue for each of the L sources streams. L is an integer greater than orequal to one. For example, a point arbitrary close to the SW sum ratelimit may be selected. In various embodiments, some of the methodelements shown may be performed concurrently, in a different order thanshown, or may be omitted. Additional method elements may also beperformed as desired. As shown, this method may operate as follows.

In 506, L submatrices of a given generator matrix G may be identified.The L submatrices may be disjoint submatrices each having the samenumber of columns as the matrix G. The numbers of rows in the Lsubmatrices of the generator matrix G are determined by the selectedpoint in the SW admissible rate region. This process of identifying Lsubmatrices of the generator matrix G is also referred to aspartitioning the generator matrix G. See below for further descriptionof how these submatrices are identified.

In 508, L parity matrices H₁, H₂, . . . , H_(L) may be computed from thegenerator matrix G. Each parity matrix H_(i) is computed from acorresponding submatrix of the generator matrix G. The parity matrixH_(i), i=1, 2, . . . , L, defines a corresponding encoder C_(i)according to the relation: (s_(i))^(T)=H_(i)(x_(i))^(T), wherein x_(i)represents a block of samples from the corresponding source stream,wherein s_(i) represents a result of the encoder C_(i).

FIG. 5B—Method for Slepian-Wolf Encoding of Multiple Data Sources

FIG. 5B illustrates one embodiment of a method for operating Ltransmitters in order to encode L respective source streams, where L isan integer greater than or equal to two. In various embodiments, some ofthe method elements shown may be performed concurrently, in a differentorder than shown, or may be omitted. Additional method elements may alsobe performed as desired. As shown, this method may operate as follows.

In 510, each of the transmitters TX_(i), i=1, 2, . . . , L, receives acorresponding parity matrix H_(i) (computed as described above). Seedescription below for more definition of the parity matrices.

In 512, each transmitter of the L transmitters encodes data from acorresponding one of the source streams using the corresponding paritymatrix H_(i). For example, each transmitter may encode data of acorresponding source stream according to the relation:(s_(i))^(T)=H_(i)(x_(i))^(T) wherein x_(i) represents a block of samplesfrom the corresponding source stream, wherein s_(i) represents a resultof the encoding.

FIG. 5C—Method for Decoding Slepian-Wolf Encoded Data from Multiple DataSources

FIG. 5C illustrates one embodiment of a method for decoding L compressedstreams of information, where L is greater than or equal to two. Invarious embodiments, some of the method elements shown may be performedconcurrently, in a different order than shown, or may be omitted.Additional method elements may also be performed as desired. As shown,this method may operate as follows.

In 514, a receiver may receive L codewords s₁, s₂, . . . s_(L) (e.g.,from L respective transmitters). The L codewords represent data from Linformation sources respectively.

In 516, the receiver generates L expanded syndromes (also referred toherein as t₁, t₂, . . . , t_(L)) from the codewords s₁, s₂, . . . ,s_(L) by inserting zero or more zero values at appropriate locations(see discussion below) into each codeword, so that each of the expandedsyndromes have the same length.

In 518, the receiver computes a vector sum of the expanded syndromes.

In 520, the receiver determines a composite codeword c closest to thevector sum (e.g., in the sense of Hamming distance).

In 522, the receiver multiplies each of L portions of a systematic partof the composite codeword c by a corresponding submatrix of a generatormatrix G to obtain a corresponding intermediate vector; thus, Lintermediate vectors are obtained altogether.

In 524, the receiver adds each of the L intermediate vectors to acorresponding one of the expanded syndromes to obtain a correspondingoutput representing an estimate of the corresponding source data.

Slepian-Wolf Coding

Various embodiments of the present invention provide a clear anddetailed solution to the problem of practical implementation ofSlepian-Wolf codes. More specifically, the approach is based onsystematic codes so that advanced channel codes can be employed to yieldSlepian-Wolf (SW) codes that can approach any point on the theoreticalbound. Additionally, practical low-complexity code designs based onpowerful systematic channel codes are described. In A. Liveris, Z.Xiong, and C. Georghiades, “Compression of binary sources with sideinformation at the decoder using LDPC codes,” IEEE CommunicationsLetters, vol. 6, pp. 440-442, October 2002, it was shown that withlow-density parity-check (LDPC) codes it is possible to approach thetheoretical limits in the SW asymmetric scenario. Irregularrepeat-accumulate (IRA) codes (see H. Jin, A. Khandekar, and R McEliece,“Irregular repeat-accumulate codes,” Proc. of 2nd InternationalSymposium on Turbo codes and related topics, pp. 1-8, September 2000,incorporated by reference above.) are a special form of LDPC codes whichsuffer very small performance loss, but can easily be coded insystematic form and have low encoding complexity which make themsuitable for multiterminal coding (see T. Berger, “Multiterminal sourcecoding”, The Information Theory Approach to Communications, G. Longo,Ed., New York: Springer-Verlag, 1977, incorporated by reference above.).Accordingly, IRA codes have been used in experiments described herein.

Additionally, to illustrate an exemplary implementation of the presentscheme with convolutional codes, powerful turbo codes (see C. Berrou, A.Glavieux, and P. Thitimajshima, “Near Shannon limit error-correctingcoding and decoding: Turbo codes,” Proc. ICC'93, IEEE Int. Conf. onComm., pp. 1064-1070, Geneva, 1993, incorporated by reference above.)are also treated. Turbo codes have already been successfully applied toasymmetric SW and Wyner-Ziv (see A. D. Wyner and J. Ziv, “Therate-distortion function for source coding with side information at thedecoder”, IEEE Trans. on Information Theory, vol. IT-22, pp. 1-10,January 1976, incorporated by reference above.) coding of two sources.Good results are obtained with both conventional (see, e.g., A. Liveris,Z. Xiong, and C. Georghiades, “Distributed compression of binary sourcesusing convolutional parallel and serial concatenated convolutionalcodes,” Proc. DCC-2003, Data Compression Conference, pp. 193-205,Snowbird, Utah, March 2003; and J. Chou, S. S. Pradhan and K.Ramchandran, “Turbo and trellis-based constructions for source codingwith side information,” Proc. DCC-2003, Data Compression Conference, pp.33-42, Snowbird, Utah, March 2003, both of which were incorporated byreference above.) and nonconventional turbo schemes (see, e.g., A. Aaronand B. Girod, “Compression of side information using turbo codes,” Proc.DCC-2002, Data Compression Conference, pp. 252-261, Snowbird, Utah,April 2002; J. Garcia-Frias and Y. Zhao, “Compression of correlatedbinary sources using turbo codes,” IEEE Communications Letters, vol. 5,pp. 417-419, October 2001; and J. Bajcy and P. Mitran, “Coding for theSlepian-Wolf problem with turbo codes,” Proc. IEEE Globecom-2001, vol. 2pp. 1400-1404, San Antonio, Tex., November 2001, each of which wereincorporated by reference above.). Various embodiments of the presentinvention implement symmetric SW coding using conventional puncturedturbo codes.

Also presented herein is an extension of the method (see S. S. Pradhanand K. Ramchandran, “Generalized coset codes for symmetric distributedsource coding,” included herewith as Appendix G; and B. Rimoldi and R.Urbanke, “Asynchronous Slepian-Wolf coding via source-splitting”, Proc.ISIT-1997 IEEE Int. Symp. Information Theory, pp. 271, Ulm, Germany,June, 1997, incorporated by reference above.) to SW coding of multiplesources (see, e.g., T. Cover, “A proof of the data compression theoremof Slepian and Wolf for ergodic sources”, IEEE Trans. on InformationTheory, vol. IT-21, pp. 226-228, March 1975, incorporated by referenceabove.), which is of special importance in sensor networks (and wirelessvideo distribution, among other application domains). For example, afterquantization of an observed corrupted version of the source, eachdistinct sensor may encode its observation by exploiting the correlationbetween the observations and the source (see Y. Oohama, “TheRate-Distortion Function for the Quadratic Gaussian CEO Problem,” IEEETrans. on Information Theory, vol. 44, pp. 1057-1070, May 1998,incorporated by reference above.).

Thus, to reach the theoretical limits (see, Y. Oohama, cited above), acode for lossless compression capable. of trading-off transmission ratesamong sensors is needed. It is shown herein that as long as thecorrelation among the sources is such that their sum is a Bernoulli-pprocess, a single channel code can be used to approach the joint entropylimit. In addition, the complexity of encoding/decoding does not exceedthat of the asymmetric codes. Furthermore, in contrast to the asymmetriccodes, the obtained code has additional error detection capability.

Below, a method for designing a single code for SW coding of multiplesources is first described, then how this theoretical approach can beapplied to practical code constructions using systematic IRA and turbocodes. Finally, experimental results for two sources and conclusions areprovided.

Multiple Source Slepian-Wolf Coding

Consider an SW coding system which consists of L encoders and a jointdecoder. Let X₁, . . . , X_(L) be discrete, memoryless, uniformlydistributed correlated random sources and let x₁, . . . , x_(L) denotetheir realizations. The i-th encoder compresses X_(i) at rate R_(i)independently from the information available at other encoders. Thedecoder receives the bitstreams from all the encoders and jointlydecodes them. It should reconstruct all received source messages witharbitrarily small probability of error. The achievable rate region isthen (see T. Cover, “A proof of the data compression theorem of Slepianand Wolf for ergodic sources”, IEEE Trans. on Information Theory, vol.IT-21, pp. 226-228, March 1975.):R _(i) ₁ + . . . +R _(i) _(k) ≦H(X _(i) ₁ . . . X _(i) _(k) |X _(j) ₁ .. . X _(j) _(L-k) )where for k≦L, {i₁, . . . , i_(k)}⊂{1, . . . , L}, and {j₁, . . . ,j_(L-k)}={1, . . . , L}\{i₁, . . . , i_(k)}.

A practical code may be constructed that can potentially approach theabove bound for any achievable rate allocation among the encoders. Thebinary case is treated, where it is assumed that all X_(i)'s are oflength n bits.

Definition 1 A general SW code is a pair (C, M), where C is an (n, k)linear binary channel code given by generator matrix G_(k×n), and M isan ordered set of integers {m₁, . . . , m_(L)} such that Σ_(j=1) ^(L)m_(j)=k.

For each i=1, . . . , L, code C_(i) may be formed as a subcode of C withgenerator matrix G_(i) _(mi×n) which consists of m_(i) rows of Gstarting from row m₁+ . . . +m_(i−1)+1. Without loss of generalitysuppose that the code C is systematic. Let m_(i−)=m₁+ . . . +m_(i−1) andm_(i+)=m_(i+1)+ . . . +m_(L). I_(k) denotes the k×k identity matrix, andO_(k1×k2) is the k₁×k₂ all-zero matrix. Then, for G=[I_(k)P_(k×(n−k))],the generator matrix of subcode C_(i) isGi=[O _(mi×mi−) I _(mi) O _(mi×mi+) P _(i) _(mi×(n−k)) ]  (1)where P^(T)=[P₁ ^(T) . . . P_(L) ^(T)].

One choice for the (n−m_(i))×n parity matrix H_(i) of C_(i) is$\begin{matrix}{H_{i} = {\begin{bmatrix}I_{m_{i\text{-}}} & O_{m_{i\text{-}} \times m_{i}} & O_{m_{i\text{-}} \times m_{i +}} & O_{m_{i\text{-}} \times {({n - k})}} \\O_{m_{i +} \times m_{i\text{-}}} & O_{m_{i +} \times m_{i}} & I_{m_{i +}} & O_{m_{i +} \times {({n - k})}} \\O_{{({n - k})} \times m_{i\text{-}}} & P_{i}^{T} & O_{{({n - k})} \times m_{i +}} & I_{n - k}\end{bmatrix}.}} & (2)\end{matrix}$Encoding may be performed by multiplication of the incoming n-lengthvector x_(i)=[u_(i) a_(i) v_(i) q_(i)] (vectors u_(i), a_(i), v_(i), andq_(i) are of length m_(i−), m_(i), m_(i+), and n−k, respectively) withthe parity matrix H_(i). In this way the syndrome vector s_(i)^(T)=H_(i)x_(i) ^(T) of length n−m_(i) may be formed as: $\begin{matrix}{{s_{i}^{T} = \begin{bmatrix}u_{i}^{T} \\\upsilon_{i}^{T} \\{q_{i}^{T} \oplus {P_{i}^{T}a_{i}^{T}}}\end{bmatrix}},} & (3)\end{matrix}$where ⊕ denotes addition in GF(2).

Let a length n row-vector t_(i) be defined as $\begin{matrix}{t_{i}^{T} = {\begin{bmatrix}u_{i}^{T} \\O_{m_{i} \times 1} \\\upsilon_{i}^{T} \\{q_{i}^{T} \oplus {P_{i}^{T}a_{i}^{T}}}\end{bmatrix}.}} & (4)\end{matrix}$Then, x_(i)⊕t_(i)=a_(i)G_(i) is a valid codeword of C_(i), and thus alsoof C. The decoder collects all syndromes s₁, . . . , s_(L) and forms thesum t₁⊕ . . . ⊕t_(L). From linearity, it follows that x₁⊕t₁⊕ . . .⊕x_(L)⊕t_(L) is a valid codeword of C. The task of the decoder is thento find a codeword c that is closest (in Hamming distance) to the vectort₁⊕ . . . ⊕t_(L). Let the vector [â₁ . . . â_(L)] be the systematic partof the codeword c. The sources may be recovered as: {circumflex over(x)}_(i)=â_(i)G_(i)⊕t_(i).

Given the length of the messages n, the number of encoders L, and theset of desirable transmission rates R₁, . . . , R_(L) (that areachievable; see T. Cover, “A proof of the data compression theorem ofSlepian and Wolf for ergodic sources”, IEEE Trans. on InformationTheory, vol. IT-21, pp. 226-228, March 1975.), parameters of the SW codemay be selected in the following way:

For i=1, . . . , L, m_(i)=n−R_(i), k=Σ_(j=1) ^(L)m_(j). If the jointdistribution of random variables X₁, . . . X_(L) is such that w(x₁⊕ . .. ⊕x_(L))≦t_(i), where w(•) denotes the Hamming weight, then the code Cshould be an (n, k, dH) code that can correct at least t errors; thus,the Hamming distance of the code is dH≧2t+1, and from the sphere packingbound n−k≧logΣ_(j=0) ^(t)(

) must hold.

Proposition 1 If the parameters of a general SW code (C,M) are selectedas above and the correlation of the sources is such that w(x₁⊕ . . .⊕x_(L))≦t, then the decoding error equals zero.

Proof: The proof follows directly from S. S. Pradhan and K. Ramchandran,“Distributed source coding: symmetric rates and applications to sensornetworks,” Proc. DCC-2000, Data Compression Conference, pp. 363-372,Snowbird, Utah, March 2000, incorporated by reference above, and thediscussion above.

An advantage of this technique is that only one good channel code isneeded. Indeed, for L=2, if the binary code C is approaching thecapacity of a binary symmetric channel (BSC), then the general SW code(C,A) will approach the SW limit as long as the joint correlationbetween X₁ and X₂ can be modeled with the same BSC. However, in the caseL>2, finding a channel that models the correlation among sources is moreinvolved. As long as this correlation is such that X₁⊕ . . . ⊕X_(L) is aBernoulli-p process, a single channel code C can be efficientlydesigned. This can be the case in the remote multiterminal setting (T.Berger, “Multiterminal source coding”, The Information Theory Approachto Communications, G. Longo, Ed., New York: Springer-Verlag, 1977,incorporated by reference above.) where an encoder observes only a noisyversion of the source. Indeed, for the source S, an observation can beoften modeled as X_(i)=S+N_(i), (i=1, . . . , L), where N_(i) is anindependent and identically distributed (i.i.d.) discrete randomvariable independent of S.

The method may also apply to the case when C is a convolutional code, aswill be shown below in an example using punctured turbo codes. Forclarity, an example of the code construction for the case L=2 using asystematic channel code (a similar example but with a non-systematiccode is hinted in S. S. Pradhan and K. Ramchandran, “Generalized cosetcodes for symmetric distributed source coding,” included herewith asAppendix G; and S. S. Pradhan and K. Ramchandran, “Distributed sourcecoding: symmetric rates and applications to sensor networks,” Proc.DCC-2000, Data Compression Conference, pp. 363-372, Snowbird, Utah,March 2000, incorporated by reference above) is presented. Let X and Ybe two discrete memoryless uniformly distributed variables of lengthseven bits such that the Hamming distance between them is at most one.The source messages are separately encoded and sent to a joint decoder.The decoder then attempts to losslessly reconstruct both sources.

The SW bound for this case is 10 bits (see D. Slepian and J. K. Wolf,“Noiseless coding of correlated information sources,” IEEE Trans. OnInformation Theory, voL IT-19, pp. 471-480, July 1973, incorporated byreference above). This bound can be achieved in the asymmetric scenarioby transmitting one source, e.g., X, at rate R₁=H(X)=7 bits and bycoding the second source, Y, at R₂=H(Y|X)=3 bits. It is shown how thesame total rate can be achieved with the symmetric approach by usingR₁=R₂=5 bits. Since n=7 bits, and a code is desired that can correct atleast one bit error, for an SW code C the systematic (7,4) Hamming codeis selected, defined by the generator matrix:$G_{k \times n} = {\left\lbrack {I_{4}P} \right\rbrack = {\begin{bmatrix}1 & 0 & 0 & 0 & 1 & 0 & 1 \\0 & 1 & 0 & 0 & 1 & 1 & 0 \\0 & 0 & 1 & 0 & 1 & 1 & 1 \\0 & 0 & 0 & 1 & 0 & 1 & 1\end{bmatrix}.}}$Its parity matrix is: $H = {\begin{bmatrix}1 & 1 & 1 & 0 & 1 & 0 & 0 \\0 & 1 & 1 & 1 & 0 & 1 & 0 \\1 & 0 & 1 & 1 & 0 & 0 & 1\end{bmatrix}.}$

Further two subcodes of C, C₁ and C₂, may be constructed by splitting Ginto two generator matrices, G₁ that contains the first m=2 rows of G,and G₂ that contains the last two rows. X may be coded using C, and Yusing C₂. Let P^(T)=[P₁ ^(T) P₂ ^(T)]. Then for the (n−m)×n parity-checkmatrices H₁ and H₂ of C₁ and C₂, respectively, the following may beobtained from (2): $\begin{matrix}{H_{1} = \begin{bmatrix}O_{m \times m} & I_{m} & O_{m \times {({n - k})}} \\P_{1}^{T} & O_{{({n - k})} \times m} & I_{n - k}\end{bmatrix}} \\{= \begin{bmatrix}O_{m \times m} & I_{n - m} \\P_{1}^{T} & \quad\end{bmatrix}} \\{{= \begin{bmatrix}0010000 \\0001000 \\1100100 \\0100010 \\1000001\end{bmatrix}},}\end{matrix}$ $\begin{matrix}{H_{2} = \begin{bmatrix}I_{m} & O_{m \times m} & O_{m \times {({n - k})}} \\O_{{({n - k})} \times m} & P_{2}^{T} & I_{n - k}\end{bmatrix}} \\{= \begin{bmatrix}1000000 \\0100000 \\0010000 \\0011010 \\0011001\end{bmatrix}}\end{matrix}$since both H₁ and H₂ have rank n−m and H₁G₁ ^(T=H) ₂G₂ ^(T)=O_(n−m)×m.)

Let realizations of the sources be x=[0 0 1 0 1 1 0] and y=[0 1 1 0 1 10]. Since the Hamming distance between x and y is one, it should bepossible to decode the messages correctly.

Syndromes for both x and y may be formed. To do so, x and y may bewritten in the formx=[a₁ v₁ q₁]=[00 10 110],y=[u₂ a₂ q₂]=[01 10 110].

The length n−m syndromes, s₁ and s₂, formed by the two subcodes are$\begin{matrix}{s_{1}^{T} = {H_{1}x^{T}}} \\{= \begin{bmatrix}\upsilon_{1}^{T} \\{{P_{1}^{T}a_{1}^{T}} \oplus q_{1}^{T}}\end{bmatrix}} \\{= \lbrack 10110\rbrack^{T}}\end{matrix}$ $\begin{matrix}{s_{2}^{T} = {H_{2}y^{T}}} \\{= \begin{bmatrix}u_{2}^{T} \\{{P_{2}^{T}a_{2}^{T}} \oplus q_{2}^{T}}\end{bmatrix}} \\{= {\lbrack 01001\rbrack^{T}.}}\end{matrix}$

The length n row-vectors t₁ and t₂ may then be given by $\begin{matrix}{t_{1}^{T} = \begin{bmatrix}O_{m \times 1} \\\upsilon_{1}^{T} \\{{P_{1}^{T}a_{1}^{T}} \oplus q_{1}^{T}}\end{bmatrix}} \\{= \lbrack 0010110\rbrack^{T}}\end{matrix}$ $\begin{matrix}{t_{2}^{T} = \begin{bmatrix}u_{2}^{T} \\O_{m \times 1} \\{{P_{2}^{T}a_{2}^{T}} \oplus q_{2}^{T}}\end{bmatrix}} \\{= {\lbrack 0100001\rbrack^{T}.}}\end{matrix}$Then the row-vectors x⊕t₁ and y⊕t₂ are codewords of the codes C₁ and C₂,respectively.

Thus, by sending s₁ and s₂ from the two encoders to the joint decoder,the decoder may find the codeword in C that is closest tot₁⊕t₂=[0110111]. Since there is no error in decoding, this codeword maybe x⊕t₁⊕y⊕t₂=[0010111] because the Hamming distance between x and y isone and the minimal Hamming distance of the code C is three. Thecorresponding reconstructions â₁=a₁ and â₂=a₂ may then be obtained asthe systematic part of the codeword. Since a₁G₁=x⊕t₁ and a₂G₂=y⊕t₂, thesources may be reconstructed as {circumflex over(x)}=â₁G₁⊕t₁=[0010110]=a₁G₁⊕t₁, ŷ=â₂G₂⊕t₂=[0110110]=a₂G₂⊕t₂. It may thusbe seen that x and y are indeed recovered error-free.

Practical Code Design

Practical SW codes using systematic IRA and turbo codes may be designedas described below using the notation established above.

Systematic IRA Codes

The present methods may be applied to systematic IRA codes (see H. Jin,A. Khandekar, and R McEliece, “Irregular repeat-accumulate codes,” Proc.of 2nd International Symposium on Turbo codes and related topics, pp.1-8, September 2000, incorporated by reference above.). Systematic IRAcodes are powerful channel codes that combine the advantages of LDPCcodes (message passing iterative decoding, simple analysis and codedesign) and turbo codes (linear time encoding). Their performance iscomparable to that of irregular LDPC codes of the same codeword length.For simplicity, symmetric SW coding of two binary sources X and Y areconsidered. Code construction for the general case is essentially thesame.

FIG. 6—Encoding Multiple Data Sources

FIG. 6 illustrates one embodiment of encoding of a source x. As FIG. 6shows, in this example, at each check (square) node 604 all theconnected information nodes 602 (cycles on the left) are modulo-2 addedand corresponding values of the parity nodes 606 (cycles on the right)are determined. Then, q₁ is modulo-2 added. Here n=10, k=6, m=3,λ(x)=0.25x+0.75x², and ρ(x)=x³.

At the first encoder, the length n source output x is split into threeparts in the formx=[a₁ v₁ q₁]  (5)

where a₁, v₁ are row-vectors of length m=k/2 and q₁ is a row-vector oflength n−k=n−2m.

First, a₁P₁ may be determined by setting the values of the systematicIRA variable nodes to [a₁ O_(1×m)], that is, half of the systematic partmay be set to zero.

Next, the length n−m syndrome s₁ that is formed by the first encoder maybe obtained by appending v₁ to u₁P₁⊕q₁. The encoding procedure isrepresented in FIG. 6.

In a similar way, s₂ may be formed at the second encoder from y=[u₂ a₂q₂]. At the joint decoder, first, vectors t₁ and t₂ may be formed asexplained above; then, a common IRA decoding of t₁⊕t₂ may be performed,and â₁ and â₂ obtained as the systematic part of the recovered codeword;finally, {circumflex over (x)} and ŷ may be reconstructed as:{circumflex over (x)}=[â ₁ O _(1×m) ]G⊕t ₁  (6)ŷ=[O _(1×m) â ₃ ]G⊕t ₂  (7)

As a result, if the used systematic IRA code can approach the capacityof a channel, then if the same channel models the statistics of x⊕y, theresulting IRA coding scheme based on the above setup will also approachthe SW limit for any rate allocation between the encoders. The procedurecan be generalized to any asymmetric scenario with any number ofsources. However, when more than two sources are used, modeling theexact correlation with a channel is more involved and hence morechallenging.

Turbo Codes

The SW code construction with systematic turbo codes (see C. Berrou, A.Glavieux, and P. Thitimajshima, “Near Shannon limit error-correctingcoding and decoding: Turbo codes,” Proc. ICC'93, IEEE Int. Conf. onComm., pp. 1064-1070, Geneva, 1993, incorporated by reference above.) isnow briefly explained. Although turbo codes consist of two convolutionalcoders, they can be treated as linear block codes. Thus, the techniquedescribed above may be applied without modification. Indeed, assumingagain the symmetric scenario, for the source realization x given by (5),a₁P₁ may be determined by coding the k-length vector [a₁ O_(1×m)] withthe first convolutional encoder. The vector [a₁ O_(1×m)] may also beinterleaved and fed into the second encoder. The syndrome may be formedthen as:s ₁ =[v ₁ a ₁ P ₁ ⊕q ₁]^(T).

To get â₁ and â₂ at the decoder, iterative maximum a posteriori decodingmay be applied to the vector t₁⊕t₂ from (4). Then, {circumflex over (x)}and ŷ may be obtained from (6) and (7), respectively.

FIGS. 7 & 8—Results

A simulation of SW coding of two i.i.d. binary discrete sources X and Ywhose correlation is modeled as a BSC with crossover probability p wasconducted. Experimental results for IRA and turbo codes are providedbelow.

In these experiments, the used systematic (n, k) IRA code is with rate0.50227 and the degree distribution polynomials are (see H. Jin, A.Khandekar, and R McEliece, “Irregular repeat-accumulate codes,” Proc. of2nd International Symposium on Turbo codes and related topics, pp. 1-8,September 2000, incorporated by reference above.):λ(x)=0.252744x²+0.081476x¹¹+0.327162x¹²+0.184589x⁴⁶+0.154029x⁴⁸,ρ(x)=x⁸. The number of iterations in the decoder was limited to 200.

The turbo encoder includes two identical recursive systematicconvolutional encoders (from W. E. Ryan, “A Turbo Code Tutorial,”included herewith as Appendix F) with memory length 4, generators (31,27) octal, and code rate 1/3. The parity bits of both encoders werepunctured to achieve the code rate of 1/2. A maximum a posteriorialgorithm was used for decoding, with the number of iterations limitedto 20.

Obtained results are shown in FIG. 7. The SW bound is 1.5 bits. Theinformation block length was k=104 and k=105 bits. For each point atleast 10⁸ codeword bits were simulated. The results are given asresidual bit error rate (BER) averaged over the two sources as afunction of the joint entropy H(X, Y)=H(X)+H(X|Y)=1+H(p).

FIG. 7 illustrates BER averaged over the two sources as a function ofthe joint entropy H(X, Y)=1+H(p) for two different information blocklengths k and two different channel coders. It can be seen that similarperformances were obtained with both coders. With the length of k=10 thegap to the SW limit was about 0.04 bits, which is comparable to theresults of the asymmetric approach with LDPC reported in A. Liveris, Z.Xiong, and C. Georghiades, “Compression of binary sources with sideinformation at the decoder using LDPC codes,” IEEE CommunicationsLetters, vol. 6, pp. 440-442, October 2002. Note that according to thepresent coding procedure, usually either both sources are recoverederror-free or both are corrupted. Also, because of the additionalencodings at the decoder side, the errors propagate. Thus, either thewhole messages are perfectly reconstructed or they are heavily damaged.(This is the reason why the drop for k=10 with IRA codes was not sharpas expected.) Therefore, the decoder can detect errors with highcertainty by comparing the two reconstructions.

FIG. 8 illustrates results with IRA codes and k=105 together with the SWbound. More specifically, three different rate allocations among theencoders were simulated by changing the number of rows (m₁ and m₂) inthe generator matrices of subcodes assigned to two encoders. In additionto the symmetric scenario, where m₁=m₂=k/2, with obtained equal rates ofboth encoders, R₁=R₂=(n−k/2)/n , two asymmetric cases were also treated.In the first case, m₁=k/3 and m₂=2k/3, resulting in R₁=(n−k/3)/n andR₂=(n−2k/3)/n. Finally, in the totally asymmetric scenario, m₁ was setto zero, and m₂ to k, which resulted in R₁=H(X)=1 and R₂=H(Y|X)=(n−k)/n.Results obtained with the IRA based scheme and k=105 together with theSW bound are shown in FIG. 8. Error-free transmission was assumed if BERwas lower than 10⁻⁶. As expected, all three cases resulted in the samegap of 0.039 bits to the bound. Thus, the different rate allocations didnot affect the performance. Similar results were obtained with thepunctured turbo coder.

CONCLUSIONS AND BENEFITS

Thus, based on the above precise and detailed interpretation of Pradhanand Ramchandran's outlined method for constructing a single channel codethat achieves arbitrary rate allocation among two encoders in the SWcoding problem (see S. S. Pradhan and K. Ramchandran, “Generalized cosetcodes for symmetric distributed source coding,” included herewith asAppendix G; and S. S. Pradhan and K. Ramchandran, “Distributed sourcecoding: symmetric rates and applications to sensor networks,” Proc.DCC-2000, Data Compression Conference, pp. 363-372, Snowbird, Utah,March 2000, incorporated by reference above.), based on the systematicsetup, a low-complexity coding designs using advanced systematic IRA andturbo codes that are capable of approaching any point on the SW boundhas been provided.

Additionally, these results were extended to SW coding of multiplesources (see T. Cover, “A proof of the data compression theorem ofSlepian and Wolf for ergodic sources”, IEEE Trans. on InformationTheory, vol. IT-21, pp. 226-228, March 1975, incorporated by referenceabove.). It has been shown herein that for a particular correlationmodel among sources, a single code can be designed, which is animportant advantage of the present method, as a single code can be usedto approach the joint entropy limit. Note that if the designed codeapproaches the capacity of the channel that models correlation, then thesystem will approach the theoretical limit. Thus, even when the numberof sources is high, since all the sources are decoded by a single code,only one (good) code is needed. In addition, low complexity and theinherent error detection capability make the present method beneficialand desirable for both direct and remote multiterminal problems (see T.Berger, “Multiterminal source coding”, The Information Theory Approachto Communications, G. Longo, Ed., New York: Springer-Verlag, 1977,incorporated by reference above.).

It is noted that to approach the theoretical limits in multiterminalcoding with a fidelity criterion, after quantization of the sources,lossless coding may be needed to further decrease the rate (see S. S.Pradhan and K. Ramchandran, “Distributed source coding using syndromes(DISCUS): design and construction,” Proc. DCC-1999, Data CompressionConference, pp. 158-167, Snowbird, Utah, March 1999; J. Chou, S. S.Pradhan and K. Ramchandran, “Turbo and trellis-based constructions forsource coding with side information,” Proc. DCC-2003, Data CompressionConference, pp. 33-42, Snowbird, Utah, March 2003; and Y. Yang, S. Chen,Z. Xiong, and W. Zhao, “Wyner-Ziv coding based on TCQ and LDPC codes,”Proc. of 37th Asilomar Conference on Signals, Systems, and Computers,Pacific Grove, Calif., November 2003, all of which were incorporated byreference above). Hence, in some embodiments, the method proposed hereinmay be applied in this second compression step. Therefore, the design ofa single practical code for an entire multi-source system, e.g., a wholesensor network, that can approach or even reach the theoretical limitsis feasible.

Although the embodiments above have been described in considerabledetail, numerous variations and modifications will become apparent tothose skilled in the art once the above disclosure is fully appreciated.It is intended that the following claims be interpreted to embrace allsuch variations and modifications.

1. A method comprising: (a) selecting any point in the Slepian-Wolf (SW)admissible rate region, wherein the point includes one rate value foreach of L correlated source streams, wherein L is greater than or equalto two; (b) identifying L submatrices of a given generator matrix G,wherein the numbers of rows in the L submatrices of the generator matrixG are determined by the selected point in the SW admissible region; (c)computing L parity matrices H₁, H₂, . . . , H_(L) from the generatormatrix G, wherein each parity matrix H_(i) is computed from thecorresponding submatrix of the generator matrix G; wherein the paritymatrix H_(i), i=1, 2, . . . , L, defines a corresponding encoder C_(i)according to the relation (s_(i))^(T)=H_(i)(x_(i))^(T), wherein x_(i)represents a block of samples from the corresponding source stream,wherein s_(i) represents a result of the encoder C_(i).
 2. The method ofclaim 1 further comprising: providing the L parity matrices to Ltransmitters respectively, each of the L transmitters multiplying thecorresponding source stream block x_(i) by the corresponding paritymatrix H_(i) to determine the result s_(i) for the encoder C_(i).
 3. Themethod of claim 1 further comprising: (a) providing the matrix G to areceiver; (b) the receiver receiving the results s₁, s₂, . . . , s_(L)from L transmitters respectively; (c) generating L expanded syndromesfrom the results s₁, s₂, . . . , s_(L) by inserting zero or more zerovalues at appropriate positions into each result, so that each of theexpanded syndromes have the same length; (d) computing a vector sum ofthe expanded syndromes; (e) determining a codeword c closest to thevector sum; (f) multiplying each of L portions of a systematic part ofthe codeword c by a corresponding submatrix of the generator matrix G todetermine a corresponding intermediate vector; (g) adding each of the Lintermediate vectors to a corresponding one of the expanded syndromes toobtain a corresponding output representing an estimate of thecorresponding source data block x_(i).
 4. A method comprising: encodingL correlated information sources using, respectively, L distinctsubmatrices of a parity check matrix, in order to generate L syndromes;sending the L syndromes to a joint decoder; the joint decoder receivingthe L compressed syndromes and decoding the L syndromes using,respectively, L submatrices of a generator matrix G; wherein thesubmatrices of the matrix G have row ranks determined by a pointselected anywhere in the Slepian-Wolf admissible rate region, whereineach submatrix of the parity check matrix is computed from thecorresponding submatrix of the generator matrix G.
 5. Acomputer-implemented method for encoding data source coding, the methodcomprising: partitioning a generator matrix to generate a plurality ofsub-matrices corresponding respectively to a plurality of correlateddata sources, wherein the partitioning is performed in accordance with arate allocation among the plurality of correlated data sources; anddetermining a corresponding plurality of parity matrices basedrespectively on the sub-matrices, wherein each parity matrix is useableto encode correlated data for a respective correlated data source.